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Summary 


Description of Concern 


The ETMS program is a Naval officer postgraduate degree program which was initiated in the early 
1970's. It is offered through civilian universities, and results in a Masters of Arts degree in 
Education and the award of a Navy subspecialty code as an Education and Training specialist. 
Eight universities currently sponsor this program with various curricula, but in each case 
successful completion results in a MA in Education and subspecialty designation. The Chief of 
Naval Education and Training (CNET) office in Pensacola, Florida directs this program, creating 
education-skill requirements (ESR's) and then matching course offerings with Navy requirements 
to ensure a certain level of consistency throughout the varying degree requirements. The match 
between skill requirements and fleet needs is crucial to the success of the program. Graduates of 
the ETMS program are expected to go on in their Naval careers working in various positions in 
areas such as education and training, instructional design, and evaluation. There are currently 142 
graduates now in the fleet. Periodic evaluation of the program is required to ensure that the 
program continues to evolve in response to fleet needs. 


Recommendations 


he A second evaluation should be conducted targeting only and all of the XX37P coded 
officers currently in the fleet. 


ae Naval Military Personnel Command should be alerted of the underutilization of ETMS 
officers and make adjustments in the assignment procedure to ensure that subspecialists are 
given a utilization tour within the first two tours after graduation as specified by OPNAV. 


o. The follow up evaluation should be of sufficient size that only the responses of XX37P 
coded officers who have been or are in a utilization tour are used to determine the relevance 
of their education to fleet ETMS jobs. 


4, Arrangements should be made with the civilian institutions to ensure that their ETMS 
program curricula meets all of the required objectives. 


2: An optional recommendation is that an ESR course be taught at all host civilian institutions 
(or at a local base) by Navy instructors. An example of just such a course has been created 








by an XX37P coded officer and will be taught this fall at Old Dominion University. For 
more information See Appendix E. 


Rationale 


The concept of relevancy of education to job needs is of great importance to both universities and 
employers. The majority of research that supports this statement falls into two broad categories; 
graduate's evaluation of the match between curricula and required job skills, and employers 
evaluation of graduates job skills, performance, and knowledge. Studies have determined that the 
more that is known about competencies needed in careers and are taken into account in curriculum 
development the more competitive graduates could be. Educational researchers have found 
evidence that follow-up studies have great value to any college, yet they have also found that 
follow-up studies of graduates to determine their perceptions of the strengths and weaknesses of 
curricula are infrequently undertaken 


Current research demonstrates the ongoing need by both educational institutions and employers for 
continuing studies that evaluate the balance between employers needs and the curriculum provided 
for students. Such evaluations will help create a dynamic and systematic relationship between 
educational institutions and the work place. 


Implications 


There should be no major cost effects involved other than those normally accrued during the 
biannual on site curriculum evaluation. The possible exception to this being the creation of a 
"Navy ESR" course. As one ETMS officer has already created the course, it would merely 
necessitate the duplication costs and the assignment of appropriate civilians or NROTC staff to 
instruct the course at site universities. 





PROGRAM DESCRIPTION 


Background 


The Education and Training Management Subspecialty (ETMS) program is a Navy officer postgraduate 
degree program which was initiated in the early 1970's. It is offered through various civilian universities, 
and results in the completion of a Masters of Arts degree in Education and the award of a Navy subspecialty 
code as an Education and Training specialist. Eight universities currently sponsor this program with 
various curricula, but in each case successful completion results in a MA in Education and subspecialty 
designation. The Chief of Naval Education and Training (CNET) office in Pensacola, Florida directs this 
program, matching course offerings with Navy requirements to ensure a certain level of consistency 
throughout the varying degree requirements. Graduates of the ETMS program are expected to go on in 
their Naval careers working in various positions in areas such as education and training, instructional 
design, and evaluation. There are currently 142 graduates who are out in the fleet. 


Evaluation Purpose 


¢ Are the knowledge and skills defined in the Education and Training Management Subspecialty 
(ETMS) Educational Skills Requirements (ESR), acquired by program participants? 

* Are the skills and knowledge acquired with the ETMS Masters of Arts degree relevant and 
applicable to Navy education jobs in the fleet? 

¢ What did Navy officers like and dislike about the ETMS program, and what would they do to 
improve it? 

¢ And a question ultimately raised, though not addressed in this study, do Navy officers find 
certain universities’ curricula more relevant to fleet jobs than others? 

These are the questions that this evaluation hopes to provide answers and insights to. 


The primary purpose for this evaluation is to determine the level ESR knowledge acquired, and if the skills 
gained from the ETMS program are useful and applicable to actual education jobs in the fleet. Additionally, 
it is hoped that information regarding the most optimal curricula/courses (or combination of 
curricula/courses) for the ETMS program would be identified. This secondary information would allow, at 
a later time, a comparison among the various universities’ curricula for potential change and improvement. 
The evaluation project examined the opinions of ETMS graduates and their supervisors regarding their level 
of ESR knowledge acquisition, and the relevancy and applicability of their degree programs to their current 
jobs. Questionnaires and interviews were triangulated to access attitudes and opinions about the program. 
It is hoped that the summary of findings will be used to trigger further evaluation of individual university 








curricula for optimal course determination, as well as be used for by decision-making groups to modify, or 


improve the existing programs to more readily meet fleet needs. 


Issues / Concerns 


The primary audience for this evaluation is CNET. Dr. J.D. Smith, head of Civilian Institutions Programs 
at CNET is the client. The secondary audience is Navy Postgraduate School at Monterey, Civilian 
Institutions coordinator, LT Padgett. Stakeholders would be the eight universities with ETMS programs: 
George Washington University, Harvard University, Memphis State University, Old Dominion University, 
San Diego State University, Stanford University, University of West Florida and recently added University 
of Rhode Island. Current and future ETMS students will also be stakeholders. 


There are political implications concerning the various civilian university curricula being scrutinized, 
however as routine changes and upgrade of the program occur based on the Navy's requirements, potential 
problems should be minimized. This evaluation should prove to be a valuable tool for the existing program 
verification system. Both clients, CNET and Navy Postgraduate School, are very supportive and interested 
in this project. 





METHODOLOGY 





Evaluation Design 


A formative evaluation design was selected utilizing a goal oriented/decision focussed approach The goal 
oriented approach was identified because of the clearly defined objectives of the ETMS program which 
facilitate agreement scale self-report measurement. This approach was not chosen singularly, however, as 
the program is ongoing and requires decision-focussed evaluation information for potential change and or 
improvement. 


A qualitative evaluation was indicated due to the extraneous variables that could influence or invalidate 
quantitative measurements. Any tangible measurement (promotion, job selection, awards etc...) of an 
ETMS officer's competency/success in the field could not be isolated from such random variables as time in 
rank, fleet requirements, personality traits, and billet opportunities. 


Sampling 


The respondents for the questionnaire were a sample of ETMS subspecialty Naval officers currently in the 
fleet. The initial criteria for respondent sample was a maximum of two tours of duty (up to six years) after 
graduation from the program. A stratified random sample was then taken from the remaining the 96 ETMS 
officers that met the criteria listed above. The stratification process was based on rank of officers 
(Lieutenants and Lieutenant Commanders) to ensure a equally representative sample from those officers 
who met the initial criteria. Once stratified the sample was taken using the random numbers table. 40 
questionnaires were mailed out to ETMS Subspecialists. 


The respondents for the supervisors questionnaires were selected using the list of ETMS coded billets listed 
in a Navy officer billet listing publication. As only 15 billets were identified for ETMS Subspecialists only 
fifteen questionnaires were sent out, one to the supervisors at each command. 


The respondents for the interviews were randomly selected from the remaining ETMS officer population. 
Both officers were working in XX37P coded billets. Both were contacted and interviewed by phone using 
the questionnaire as a format for the closed and open response questions. 


Data Collection 


Three types of data were collected for a triangulation of information. The first type was the questionnaires 
mentioned above. These were in a semi-structured format with a predominance of agreement scale 
questions followed by open response probes at the end. The questions asked the respondents to rate their 
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fleet education jobs. The open-response portion of the questionnaires asked the sample for their personal 
ETMS history information, their opinions of the program, and their recommendations for program 


improvement. 


_ The second collection method was also agreement scale questionnaires, however these were targeted at 


fifteen supervisors that oversee ETMS officers in fleet education jobs. This questionnaire followed the 
same format as above, only supervisors rated their subordinates on their mastery of the the 10 ESR's as 
well as the supervisors rating their perceptions of skill’s relevancy to the job. It also ended with an open 
response section. 


Finally two interviews were conducted. The interview, in a highly structured format, paralleled the 
questionnaires with the emphasis on open responses to all questions. The first part of the interview used 
the same format of the agreement scale questions used above. This was followed by the same open-ended 
questions that were also used at the end of the questionnaires. 


Instrument Objectives 


The following questions were addressed by this evaluation study via the two questionnaires and the 
interviews: 
* Is the knowledge level specified by the ten educational skill areas acquired through the ETMS 
program’? 
¢ Are the skills attained from the ETMS program relevant and applicable to fleet education jobs? 
* What did Navy ETMS officers like and dislike about the program and what would they do to 


improve it? 


Additional information collected at the clients request include: 
* University attended during ETMS program participation. 
¢ How long between graduation and utilization tour. 
¢ What is more career enhancing: department head or utilization tour. 
¢ What continuing education has been pursued. 


Data Analysis 


The data collected was analyzed by two methods. The closed response data collected in the 
questionnaires was scored and rated on knowledge level, and the applicability and relevance (both graduate 
and supervisor ratings) of the 10 skill areas to education jobs. Values were assigned to responses On a 
scale of 1 to5. One being very low, five being very high. Number and percentage of responses were 
assigned to ESR's by respondent category. Mean scores were also assigned for each ESR rating by 
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respondent category. The open responses were analyzed with the those from the interviews using the 
responses themselves to create categories or patterns of response. Trends were identified and then analyzed 
against the closed responses for potential commonality. 


Limitations of Study 


The major limitation of this evaluation is it's potential lack of generalizability. This is due to the low 
number of usable questionnaires returned by the sample. Besides the normal disinterest most people have 
for completing and returning questionnaires a secondary element was identified as the cause for low 
usability of responses. The sample was selected from a computer print out titled "XX37P Coded Officers" 
that was obtained from the Graduate Education Placement Department of the Navy Military Personnel 
Command at Washington D.C. As questionnaires were returned, it became apparent that the list sent from 
Washington contained not only XX37P coded officers, but XX37S coded officers as well. The "S" code 
identifies that the individual received their subspecialty rating not from a school program, but through 
experience alone. Table 1 demonstrates the response rates. 


Table 1 
Questionnaire Response Rate 


Responses 


N=27 
ETMS Graduates 


Lieutenant Commanders 
Lieutenants 


Total 


(Usable) 





ETMS Officer Supe 


Basically despite the fact of a55% questionnaire return rate from the ETMS officers, after removing the 
XX37S officers from the returned data, only 37% of those returned were usable. Due to time restrictions it 
was not possible to request a second “purified” list of P coded officers and re-select the sample. 





FINDINGS 





The following information was collected and divided into three primary sections to address the major 
evaluation questions: Is the knowledge level specified by the ten educational skill areas acquired through 
the ETMS program?; Are the skills and knowledge attained from the ETMS program relevant and applicable 
to fleet education jobs?; What did Navy ETMS officers like and dislike about the program and what would 
they do to improve it? The additional demographic information collected at the client s request can be found 
in Appendix A. 


Is the knowledge level specified by the ten ESR's acquired through the 


ETMS program? 


Educational! skill requirements were rated by the sample based on level of knowledge attained and level of 
Televance to fleet jobs. The findings from the data were further divided by ETMS officers self rated 
information, and ETMS Officer Supervisor rated information. Finally all data collected, including 
interviews were compared in tables under each heading. 


Budget constraints and fleet needs dictate that the Navy educational programs not only identify specific 
knowledge required for a field, but ensure that it is acquired by program participants. Table 2 displays the 
rate of ESR knowledge acquired as identified by actual ETMS officers based on number and percent of 
responses to each knowledge acquisition level. The major portion of the respondents rated their knowledge 
acquisition of ESR's as moderate to high. The noticeable exceptions are ESR's 2,3, and 9. All three of 
these skill requirements (Naval Education and Training organization, Planning Programming and 
Budgeting System, and Contract Administration) are very uniquely "Navy" oriented which might help 
explain the lack of this type of education in civilian university programs. This table also illustrates the 
central tendency in scoring, with a very strong inclination for a "moderate" rating average. 
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Table 2 
Level of ESR Knowledge Acquisition 
(Self-Rated) 
N=15 


Very Very No 
High High Moderate Low Low _ Opinion 
T Wy Vp 07 





Educational Skill Requirement Level 


1. Knowledge of basic Management 
Principles: 


2. Knowledge of Naval Education and 
3. Knowledge of Planning, Programming 
and Budgeting System (PPBS): 


4. Knowledge of the principles and 


management policies and procedures: 


6. Knowledge of training/educational 
psychology: 


7. Knowledge of the Training Research and 
Development process: 


8. Knowledge of the applications of 
computer technology to Education/Training 
management: 


9. Knowledge on Contract Administration: 


10. Knowledge on existing and conceptual 
training systems: 





Table 3 quantifies the same type of data as rated by ETMS officer supervisors. 





Table 3 
Level of ESR Knowledge Acquisition 
(Supervisor Rated) 


Very Very 
High High Moderate Low Low 
70 


Educational Skill Requirement Level 


1. Knowledge of basic Management 
Principles: 


2. Knowledge of Naval Education and 
Training (NAVEDTRA) organization and 
operations: 


3. Knowledge of Planning, Programming 
and Budgeting System (PPBS): 


4. Knowledge of the principles and 
applications of Instructional Systems 
Development (ISD): 


5. Knowledge of Personnel/Manpower 
management policies and procedures: 


6. Knowledge of training/educational 
psychology: 


Development process: 


8. Knowledge of the applications of 
computer technology to Education/Training 


management: 


9. Knowledge on Contract Administration: 


10. Knowledge on existing and conceptual 
training systems: 








No 
Opinion 


This table shows that the majority of supervisors rate their ETMS officers as having a moderate to very 
high level of ESR knowledge acquisition. Only one skill area was rated below this, skill requirement 
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number 9, which was rated low to moderate. As stated before, Navy Contract Administration can be very 
specific in terms of Navy regulations which would make certain aspects of it difficult to be offered as a 
regular course at a civilian university. It is also apparent that supervisors rate the knowledge of their ETMS 
officers more highly than the actual officers themselves. Table 4 is an abbreviated summary of the number 
and percentages of the responses for all of the respondents as a comparison of rating levels. 


Table 4 
Knowledge Acquisition: Abbreviated Response Summary 
ETMS officers, Interviewees, and Supervisors 


ESR # and Respondents 


Low to Very Low 
7 (N) 


High to Very 
High 
%o (N 








ESR #1. ETMS officers 
Interviewees 
Supervisors 









. ETMS officers 
Interviewees 
Supervisors 






. ETMS officers 
Interviewees 
Supervisors 






. ETMS officers 
Interviewees 
Supervisors 






. ETMS officers 
Interviewees 
Supervisors 






ESR #6. ETMS officers 
Interviewees 
Supervisors 






ESR #7. ETMS officers 
Interviewees 
Supervisors 






ETMS officers 
Interviewees 
Supervisors 






ETMS officers 
Interviewees 
Supervisors 


ESR # 9. 









ESR # 10. ETMS officers 
Interviewees 
Supervisors 



























53.5% (8) 


100% (2) 
100% (6) 


13% (2) 
0% 
83% (5) 


6.5% (1) 
100% (2) 
67% (4) 


40% (6) 
100% (2) 
100% (6) 


47% (7) 
100% (2) 
100% (6) 


80% (12) 
100% (2) 
100% (6) 


33% (5) 
50% (1) 
83% (5) 


33% (5) 


6.5% (1) 
50% (1) 
33% (2) 


20% (3) 
30% (1) 
83% (5) 


40% (6) 
0% 
0% 


27% (4) 
50% (1) 
16.5% (1) 


20% (3) 
0% 
16.5% (1) 


20% (3) 
0% 
0% 


40% (6) 
0% 
0% 


13% (2) 
0% 
0% 


47% (7) 
50% (1) 
16.5% (1) 


40% (6) 
100% (2) 
16.5% (1) 


20% (3) 
50% (1) 
16.5% (1) 


40% (6) 
50% (1) 
16.5% (1) 












6.5% (1) 
0% 
0% 


60% (9) 
50% (1) 
0% 


67% (10) 
0% 
16.5% (1) 


40% (6) 
13% (2) 
0% 


13% (2) 
0% 
0% 


6.5% (1) 
0% 
0% 


13% (2) 
0% 
0% 


27% (4) 
0% 
16.5% (1) 


67% (10) 
0% 
67% (3) 


27% (4) 
0% 
0% 




























































0% 
0% 
0% 


0% 
0% 
0% 


6.5% (1) 
0% 
0% 


0% 
0% 
0% 


0% 
0% 
0% 


0% 
0% 
0% 


6.5% (1) 
0% 
0% 


0% 
0% 
0% 


6.5% (1) 
0% 
0% 


13% (2) 
0% 
0% 





Here by including the interviews, the wide range of response is evident. A better way to view this 
information is using the mean scores assigned for all three sample groups, as shown in Figure 1. An 
interesting feature identified here is that the interview responses are much closer in rating value to the 
supervisors than that of the rest of the ETMS officers who responded to the questionnaires. 


Figure 1 
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This figure demonstrates the vast variation of opinions on skill requirement acquisition. Because of the 
limited number of responses on which it is based, it is very difficult to identify any commonality in the 
ratings. Those skill areas with the most obvious similarity in means are ESR 6 - Education and Training 
psychology, rated high to very high, and ESR 7 - Training Research and Development Processes rated 
moderate to high. The overall mean for the ESR knowledge level is 3.6 or moderate to high. 


Are the skills and knowledge attained from the ETMS program relevant 


and applicable to fleet education jobs? 


The second major area of information collected was on the perceived level of ESR relevancy to fleet jobs. 
The first part of this discussion of findings will focus in on ESR relevancy as rated on the ETMS officers 
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questionnaires. This area of report had a great potential for variability due to the varying jobs and degrees 


of skills required in each. 
Table 5 
Level of ESR Relevancy to Fleet Job 
(Self Rated) 


N=15 
Very Very No 
High High Mocerate Low Low Opini 








Educational Skill Requirements 


1. Relevance of basic Management 
Principles to the job: 


2. Relevance of Naval Education and 
Training (NA VEDTRA) organization and 
operations to the job: 


3. Relevance of Planning, Programming 
and Budgeting System (PPBS) to the job: 


4. Relevance of the principles and 
applications of Instructional Systems 
Development (ISD) to the job: 


5. Relevance of Personnel/Manpower 
management policies and procedures to the 
job: 


6. Relevance of training/educational 
psychology to the job: 


7, Relevance of the Training Research and 
Development process to the job: 


8. Relevance of the applications of 
computer technology to Education/Training 
management to the job: 


9, Relevance on Contract Administration to 
the job: 


10. Relevance on existing and conceptual 
training systems to the job: 





This table demonstrates that for nine out of ten of the skill areas, more than half of the respondents rated 
them very highly relevant. The exception being Contract Administration which was rated by 53% as low to 
very low in job relevance. This table especially interesting with it's consistent polarity of the ETMS officer 
responses. This could be due to the fact that just less than half (as identified later on in this report) of the 
respondents had ever even worked in an XX37P (ETMS) coded billet. The overall mean average for all ten 
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skill areas was 3.12, or slightly higher than a moderate level of job relevancy. Table 6 lists the responses 


on relevancy as perceived by ETMS Officer supervisors. 
Table 6 
Level of ESR Relevancy to Fleet Job 
(Supervisor Rated) 


Very Very 
High High Moderate Low Low 


Educational Skill Requirements 


1. Relevance of basic Management 
Principles to the job: 


2. Relevance of Naval Education and 
Training (NA VEDTRA) organization and 
operations to the job: 


3. Relevance of Planning, Programming 
and Budgeting System (PPBS) to the job: 


4. Relevance of the principles and 
applications of Instructional Systems 
Development (ISD) to the job: 


5. Relevance of Personnel/Manpower 


job: 


6. Relevance of training/educational 
psychology to the job: 


7. Relevance of the Training Research and 
Development process to the job: 


8. Relevance of the applications of 
computer technology to Education/Training 
management to the job: 


9. Relevance on Contract Administration to 
the job: 


10. Relevance on existing and conceptual 
training systems to the job: 








No 


This table once again shows that Supervisors (with two exceptions) consistently rate ESR’s higher than 
their ETMS officers. The majority of the skill areas were rated at a moderate to very high level of relevance 
to fleet jobs. The overall mean score for all ten of the ESR's was 3.5 or moderate to high. The exception 
to this were ESR 7 - Training Research and Development, and ESR 8 - Computer Technology. Both were 
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rated as low in job relevance. Table 7 once again summarizes all the responses from the sample on on ESR 
relevancy to fleet jobs. 


Table 7 
ESR Relevancy to Job: Abbreviated Response Summary 
ETMS officers, Interviewees, and Supervisors 








High to Very Low to Very Low 
ESR # and Respondents High 
7. (N) 7 (N) 
























ESR #1. ETMS officers 
Interviewees 
Supervisors 


. ETMS officers 
Interviewees 
Supervisors 


. ETMS officers 
Interviewees 
Supervisors 


. ETMS officers 
Interviewees 
Supervisors 


. ETMS officers 
Interviewees 
Supervisors 


ETMS officers 
Interviewees 
Supervisors 


ETMS officers 
Interviewees 
Supervisors 


ETMS officers 
Interviewees 
Supervisors 


. ETMS officers 
Interviewees 
Supervisors 


ESR # 10. ETMS officers 
Interviewees 
Supervisors 



























73% (11) 
100% (2) 
100% (6) 


33% (5) 
50% (1) 
67% (4) 


27% (4) 
100% (2) 
50% (3) 


40% (6) 
50% (1) 
50% (3) 


80% (12) 
100% (2) 
100% (6) 


67% (10) 
100% (2) 
100% (6) 


27% (4) 
50% (1) 
0% 


60% (9) 
50% (1) 
16.5% (1) 


33% (5) 
100% (2) 
50% (3) 


20% (3) 
50% (1) 
50% (3) 









































20% (3) 
0% 
0% 


27% (4) 
0% 
16.5% (1) 


27% (4) 
0% 
0% 


6.5% (1) 
0% 
0% 


13% (2) 
0% 
0% 


20% (3) 
0% 
0% 


27% (A) 


13% (2) 
0% 
33% (2) 


6.5% (1) 


40% (6) 
O% 
16.5% (1) 


6.5% (1) 
0% 
0% 


47% (7) 
50% (1) 
16.5% (1) 


50% (3) 


53% (8) 
50% (1) 
50% (3) 


6.5% (1) 
0% 
0% 


13% (2) 
0% 


0% 


40% (6) 
50% (1) 
50% (3) 


27% (4) 
50% (1) 
50% (3) 


53% (8) 
0% 
50% (3) 


27% (4) 
50% (1) 
33% (2) 


0% 
0% 
0% 


0% 
0% 
0% 


6.5% (1) 
0% 
0% 


0% 
0% 
0% 


0% 
0% 
0% 


0% 
0% 
0% 


6.5% (1) 
0% 
0% 


0% 
0% 
0% 


6.5% (1) 
0% 
0% 


13% (2) 
0% 
0% 





This table points out that most of the respondents (again with two exceptions) rated ESR to job relevancy as 

moderate to very high. It is also more consistent and shows more commonality in opinion than the earlier 

representation on knowledge acquisition. This is even more apparent in Figure 2, which is a bar graph of 
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the ESR means for each set of respondents. This chart graphically demonstrates the particular similarity in 
ratings of most of the skill areas except ESR's 3 and 9. 


Figure 2 
Level of Relevance of Educational Skill Requirements 
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The commonality shared in the majority of ratings of the ESR's job relevance seem to point to it as being 
more universally measured by both ETMS officers and their Supervisors. The overall mean for the 10 skill 
areas 1S a 3.5, or a moderate to high relevancy factor. The two low to moderate rated areas were Training 
Research and Development (ESR 7), and Computer Technology Applications (ESR 8). 


Program Positive/Negative Comments and Recommended 
Improvements 


The final section of the findings and the questionnaire/interviews related to the perceived "good and bad” 
aspects of the ETMS program. The following tables summarize the most frequently cited positive and 
negative open-responses at the end of the questionnaires and interviews. For a complete summary of all 
responses, see Appendix C. 
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Table 8 
Positive ETMS Program Open Responses 


Interesting subject variety and selection 
Good adult learning and educational psychology courses 


Outstanding ISD instruction 
Business courses were the most valuable 
Graduates have an extensive background in Educational Training and Management 





The second and third responses are useful in that they support data collected in the agreement scale portion 

_ of the questionnaires and interviews. Response number two directly reflects the overall mean rating of 

: knowledge acquisition of high to very high in ESR 6- Training and Educational psychology. Response 
number three is interesting in that it coincides with the overall mean rating of high in knowledge acquisition 
for ESR 4. Response number five lightly contradicts the overall mean rating of moderate to high for the 
knowledge acquisition of ESR 1, Management Principles. 


Table 11 lists the most frequently cited negative responses about the ETMS program. 


Table 9 
Negative ETMS Program Open Responses 


Many course options aren't relevant to program (ie. School Law, Computer Programming, 
Client/consultant courses) 
University ETMS program advisor is not helpful 


Courses geared strictly to civilian programs/needs (primarily teachers and principals) 

No courses on Contract Administration - need for job 

Management courses redundant for Naval Officers 

Frequently courses most relevant to ESR's no offered (during program completion time) 





The comments made in response one, that courses are frequently not applicable to the program (computer 
programming is one of the courses cited), supports the overall low to moderate relevancy rating of ESR 8 - 
Computer Technology. On the other hand it does not support the low to moderate rating of knowledge 
level for the same skill area. Response four about Contract Administration supports the overall very low to 
low rating for knowledge level of ESR 9, and becomes an important statement when compared to an overall 
rating of moderate to high in job relevancy. Response number five on management courses, is balanced by 
an overall rating for ESR 1 (Management Principles) of Moderate to High in knowledge level but a high to 
very high level of relevancy. The final response is especially interesting in that it supports the low to 
moderate rated knowledge level ESR's (2- NAVEDTRA Organization, 3- Planning Programming and 
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Budgeting System and 9- Contract Administration). The last response validates the need for courses that 

-meed the curricula standards for ESR's 2,3, and 9. 

This last table summarizes the most frequently mentioned comments on changes to improve the program 
that the respondents said that they would like to see. As above a complete summary of all responses can be 
found in Appendix C. 


Table 10 
Recommended Change Responses to Improve ETMS Program 


Add courses on: 
Contract Administration 
Program evaluation 


How Navy training courses curriculum are designed and developed 
ISD (specially test development and objective writing 

Navy Training ESR course 

Educational Technology (development and training aids) 





The most frequently cited responses all involve course additions. While responses one and three are 
supported by previously mentioned low to moderate rated knowledge level ESR's, responses four and six 
are contradicted by the low to moderate job relevancy of their related ESR's. 
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CONCLUSIONS & RECOMMENDATIONS 


1, Based on the smail size of the useable 
response from the sample, this evaluation 
may not be generalizable to the entire 
ETMS program and it's community of 
subspecialists. 


2. More than half of the responses were 
received from ETMS officers who have 
never been in a utilization tour. 


3. Three Educational Skill Requirements 
(Contract Administration, Personnel 
Planning and Budgeting System, and 
NAVEDTRA Organization) are not 
sufficiently addressed by most civilian 


universities. 


A secondary evaluation should be conducted 
targeting only and all of the XX37P coded 
officers. 


Naval Military Personnel Command should be 
alerted to this and make adjustments in the 
assignment procedure to ensure that 
subspecialists are given a utilization tour within 
the first two tours after graduation as specified 
by OPNAV. 


The follow up evaluation should be of 
sufficient size that only the responses of 
XX37P coded officers who have been or are in 
a utilization tour are used to determine the 
relevance of their education to fleet ETMS jobs. 


Arrangements should be made with the civilian 
institutions to ensure that their ETMS program 
curricula meets all of the required objectives. 


An optional recommendation is that an ESR 
course be taught at all host civilian institutions 
(or at a local base) by Navy instructors. An 
example of just such a course has been created 
by and XX37P coded officer and will be taught 
this fall at Old Dominion University. For more 
information See Appendix E. 
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LITERATURE REVIEW 





The concept of relevancy of education to job needs is of great importance to both universities and 
employers. The majority of literature that supports this statement falls into two broad categories; graduate's 
evaluation of the match between curricula and required job skills, and employers evaluation of graduates 
job skills, performance, and knowledge. 


Wheelock and Zekeri (1988) conducted a study that attempted to determine which competencies former 
students found most essential to their careers. They hypothesized that the more that is known about 
competencies needed in careers and are taken into account in curriculum development the more competitive 
graduates could be. According to Cedergrin (1985) continuing follow-up has great value to any college, 
yet he feels that follow-up studies of graduates to determine their perceptions of the strengths and 
weaknesses of curricula are infrequently undertaken. In his book, Konrad (1975) expressed the opinion 
that data that are generated from follow-up studies should be utilized as soon as they are available and then 
stored in a management information system until further analysis can be undertaken. In an article by Hoyt 
(1987), results from a 1987 Gallup poll are cited that indicate only about half of the employed college 
graduates said that their skills and abilities were being used “very well” in their current jobs. Houghton's 
studies (1971) of the business program at Southern Oregon College found that the weakest aspects of the 
program in the opinion of the graduates were that the courses were too theoretical and the curriculum was 
poor. In another recent study (Gerbert 1988) surveys were used to identify the topics that new dentists 
believed to be under-emphasized in the dental school curriculum as well as those that are most important to 
dental practice. 


Carnoy (1979) researched the role of education in meeting employment objectives in the hopes of 
identifying policy oriented planning procedures to ensure employer needs are met. Also hoping that the 
results of study would help improve educational programs to better meet employers’ needs, Travis and 
Jones (1976) had supervisors evaluate graduates in the area of job performance. Ina another study to 
determine how adequately college prepared 1978 graduates for employment in some specific job skill and 
areas of knowledge, Campbell and Dogoloff (1979) found that employers rated 94% of graduates 
performance of job skills as “adequate or more than adequate." Additionally, the survey results indicated 
that while supervisors rated the overall job performance of graduates as adequate or more than adequate, 
they cited the need for improvement in technical and problem solving skills. 


These studies demonstrate the ongoing need by both educational institutions and employers for continuing 
research that evaluates the balance between employers needs and the curriculum provided for students. 
Such evaluations will help create a dynamic and systematic relationship between educational institutions and 


the work place. 
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APPENDICES 





Appendix A. - Client Requested Demographics 





This next section of findings will report the information collected specifically at the clients request. 
Current Billet 


The first question to be addressed is the utilization of ETMS officers in specific XX37 coded billets. 
‘Figure 3 is a pie Chart representation of the respondents (ETMS officers only) cited tour utilization history. 
It reveals several interesting trends. The foremost being that nearly half (41%) of the respondents have not 
had a utilization tour to date. This is especially alarming in that it identifies the underutilization of a readily 
available and costly asset: Navy officers with masters degrees in education that were frequently fully 

funded by the government. 


ETMS Officer XX37P Billet Utilization 





ea Current Tour 41% (7) 
Previous Tour 20% (3) 
Bl Never 41% (7) 





Of the 17 ETMS Officer responses (including the interviewees) six were placed in utilization tours 
immediately after graduation, four on their second tour after graduation. As noted above the remaining 
seven have never had a utilization tour. Personal preference may be a factor in this, as officers frequently 
choose billets outside of their subspecialty for various reasons. 


Universities Attended 


The following table identifies the respondents by the universities they attended during their participation in 
the the ETMS program. Only five of the possible eight universities are represented in the sample, but each 
is relatively equal in representation. 





Table 11 
Breakdown of ETMS Graduate Respondents 
by University 
Nai 





San Diego 
State 


Old Dominion 


7o(N) 


West Florida 


Stanford George 
Washington 


7o(N) 





7o(N) 


) %(N) 
| 20% (3) 27% (4) 


Continuing Education 


Of the 17 ETMS officers (including interviewees) only 2 had, or are pursuing continuing education. Of 
those two, one is taking courses in Education and Human Resources Development, the other has completed 
a doctorate in Education Administration. 


Career Enhancement 


This question is very pertinent as the Navy continues to shift back and forth from a strictly leadership track 
to one more based on a subspeciality career emphasis. It is interesting to note that the majority of both the 
ETMS officers and their supervisors cited the Department Head tour as more career enhancing. The 
comments made by the three respondents that answered out of category, were particularly helpful in 
identifying what is perceived as "the best of both worlds", the optimum answer: a dual Department 
Head/Utilization tour. Unfortunately this type of billet is considerably rare in ETMS assignments. 


Table 12 
ETMS Officer and Supervisor 
Career Concepts 
N=232 


Department Head 
Tour 


Zo _(N) 


Subspecialty Utilization 
Tour 


7% (N) 
27% (4) 
0% 
17% (1) 
35% (5) 


Most Career Enhancing 
after Graduation? 

















60% (9) 






ETMS officers 






100% (2) 
66% (4) 
52% (15) 


EIMS Offficer Interviewees 











ETMS Officer Supervisors 






Total 








* Two of the ETMS officers and one Supervisor wrote in that the "ideal career enhancing tour” after 
graduation would be a Department Head/Utilization tour. 





Appendix B. - Questionnaires 


ETMS Officer Questionnaire 


Try to remember back, and based solely on your ETMS degree education for the XX37P subspecialty, 

please rate the the following Education Skill Requirements by the level of knowledge you acquired and it's 
japplicability and relevance to your current job. Please read each skill area and circle the level that most 

closely matches yours. There are six possible responses: Very High, High, Moderate, Low, Very Low, 


and No Opinion. 


At the end there are eight general information questions, followed by space for your recommendations on 
improving the ETMS program. Feel free to add any additional comments that you feel would be pertinent. 
Thank you again for your time. 


tt SS SS SP ar OU aru 


1. Level of knowledge of basic Management Principles: 
* fundamental management theories 
* management system basic elements 
* resource management system 


2. Level of relevance of basic Management Principles to 
your job. 


3. Level of knowledge of Naval Education and Training 
(NAVEDTRA) organization and operations: 


* organizational structure 

* training program implementation procedures 

* training program maintenance 

* training requirement validation 

* education and training roles of Navy commands 


4. Level of relevance of NAVEDTRA organization and 
operations knowledge to your job. 


5. Level of knowledge of Planning, Programming and 
Budgeting System (PPBS): 


* translating validation into acquisition 
6. Level of relevance of PPBS to your job. 


7. Level of knowledge of the principles and applications of 
Instructional Systems Development (ISD): 


* Tenets of instructional technology and ISD 
¢ ISD advantages 
* instructor roles with ISD 


8. Level of relevance of ISD to your job. 





Very High 
Very High 
Very High 
Very High 


Very High 


Very High 


Very High 
Very High 
Very High 
Very High 
Very High 


Very High 


Very High 


Very High 
Very High 


Very High 


Very High 
Very High 
Very High 


Very High 


High 
High 
High 
High 


High 


High 


High 
High 
High 
High 
High 
High 
High 
High 
High 
High 
High 
High 
High 


High 


Moderate 
Moderate 
Moderate 
Moderate 


Moderate 


Moderate 


Moderate 
Moderate 
Moderate 
Moderate 
Moderate 
Moderate 


Moderate 


Moderate 
Moderate 


Moderate 


Moderate 
Moderate 
Moderate 


Moderate 


Low 


Low 


Low 


Low 


Low 
Low 


Low 





Very Low 
Very Low 
Very Low 
Very Low 


Very Low 


Very Low 


Very Low 
Very Low 
Very Low 
Very Low 
Very Low 


Very Low 


Very Low 


Very Low 
Very Low 


Very Low 


Very Low 
Very Low 
Very Low 


Very Low 


No Opinion 
No Opinion 
No Opinion 
No Opinion 


No Opinion 


No Opinion 


No Opinion 
No Opinion 
No Opinion 
No Opinion 
No Opinion 


No Opinion 


No Opimion 


No Opinion 
No Opinion 


No Opinion 


No Opinion 
No Opinion 
No Opinion 


No Opinion 





9. Level of knowledge of Personnel/Manpower management Very High High Moderate Low Very Low No Opinion 
policies and procedures: 


* theories of manpower management Very High High Moderate Low Very Low No Opinion 
* tenets of personnel administration Very High High Moderate Low Very Low No Opinion 
10. Level of relevance of basic Personnel/Manpower Very High High Moderate Low Very Low No Opinion 


management policies and procedures to your job. 

11. Level of knowledge of training/educational psychology: Very High High Moderate Low Very Low No Opinion 
* learning processes Very High High Moderate Low Very Low No Opinion 
¢ learning theory Very High High Moderate Low Very Low No Opinion 


12. Level of relevance of Training/Educational Psychology Very High High Moderate Low Very Low No Opinion 
to your job. 


13. Level of knowledge of the Training Research and Very High High Moderate Low Very Low No Opinion 
Development process: 
* steps for the research and development process Very High High Moderate Low Very Low No Opinion 


* research and development initiation and guidance Very High High Moderate Low Very Low No Opinion 


14. Level of relevance of the Training Research and Very High High Moderate Low VeryLow No Opinion 
Development process to your job. 
15. Level of knowledge of the applications of computer Very High High Moderate Low Very Low No Opinion 
technology to Education/Training management: 
* ADP systems Very High High Moderate Low Very Low No Opinion 
* computer theory and system analysis Very High High Moderate Low Very Low No Opinion 
* management of computer applications Very High High Moderate Low Very Low No Opinion 
* computer capability/limitations Very High High Moderate Low Very Low No Opinion 
* computer assisted instruction Very High High Moderate Low Very Low No Opinion 


16. Level of relevance of computer technology knowledge to Very High High Moderate Low Very Low No Opinion 
your job. 


17. Level of knowledge on Contract Administration: Very High High Moderate Low Very Low No Opinion 
* procurement management Very High High Moderate Low Very Low No Opinion 

* procurement regulations Very High High Moderate Low Very Low No Opinion 

* product evaluation Very High High Moderate Low VeryLow No Opinion 

Low Very Low No Opinion 


18. Level of relevance of Contract Administration to your Very High High Moderate 
job. 





| 19. Level of knowledge on existing and conceptual traming Very High High Moderate Low 
systems: 


* non ISD training theories: assets/limitations Very High High Moderate Low 


20. Level of relevance of knowledge on existing and Very High High Moderate Low 
conceptual training systems to your job. 


21. Are you currently in an XX37P coded billet? 
* If no, when were you last in an XX37P coded billet? 


22. What university did you attend during your participation in the ETMS program? 


23. How long was it after graduation before your XX37P utilization tour? 
24. Have you pursued continuing education since graduation? 


¢ If yes, what courses and in what field? 


Very Low 


Very Low 


Very Low 


25. What would have been more career enhancing for you after graduation; a department head or utilization tour? 


26. Was there anything that you especially liked about the curricula offered at your university for ETMS program? 


27. Was there anything that you especially disliked about the curricula offered at your university for ETMS program? 


No Opinion 


No Opinion 


No Opinion 
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28. If you could, what changes/improvements would you make to the ETMS program, or the university's curriculum to make 
it more job relevant? 


Please feel free to make any additional comments. 


—— Thank you again for your help. — —— 
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| 
ETMS Supervisor Questionnaire 
: 






Moderate, Low, Very Low, and No Opinion. 


1. Level of knowledge of basic Management Principles: 


¢ fundamental management theories 
* management system basic elements 


* resource management system 


2. Level of relevance of basic Management Principles to the 


job. 


3. Level of knowledge of Naval Education and Training 
(NAVEDTRA) organization and operations: 


* organizational structure 

* training program implementation procedures 

* training program maintenance 

¢ training requirement validation 

¢ education and training roles of Navy commands 


4. Level of relevance of NAVEDTRA organization and 
operations knowledge to the job. 


5. Level of knowledge of Planning, Programming and 
Budgeting System (PPBS): 


* translating validation into acquisition 
6. Level of relevance of PPBS to the job. 


7. Level of knowledge of the principles and applications of 
Instructional Systems Development (ISD): 


¢ Tenets of instructional technology and ISD 
° ISD advantages 
¢ instructor roles with ISD 


8. Level of relevance of ISD to the job. 


Based on your knowledge of and experience with the XX3 
him/her on the following Education Skill Requirements by the level of knowledge displayed to you. Also 
please rate the level of applicability and relevance of these skills to his/her job. Please read each skill area 
and circle the level that most closely matches. There are six possible responses: Very High, High, 






Very High 
Very High 
Very High 
Very High 


Very High 


Very High 


Very High 
Very High 
Very High 
Very High 
Very High 


Very High 


Very High 


Very High 
Very High 


Very High 


Very High 
Very High 
Very High 


Very High 


High 
High 
High 
High 


High 


High 


High 
High 
High 
High 
High 


High 


High 


High 
High 


High 


High 
High 
High 


High 


Moderate 
Moderate 
Moderate 


Moderate 


Moderate 


Moderate 
Moderate 
Moderate 
Moderate 
Moderate 
Moderate 


Moderate 


Moderate 
Moderate 
Moderate 


Moderate 
Moderate 
Moderate 


Moderate 


Low 


Low 


Low 


officer who works for you, please rate 


At the end there are six general information questions, followed by space for any recommendations you 
may have for improving the ETMS program/curricula. Feel free to add any additional comments that you 
feel would be pertinent. Thank you again for your time. 


Very Low 
Very Low 
Very Low 
Very Low 


Very Low 


Very Low 


Very Low 
Very Low 
Very Low 
Very Low 
Very Low 


Very Low 


Very Low 


Very Low 
Very Low 


Very Low 


Very Low 
Very Low 
Very Low 


Very Low 

















No Opinion 
No Opinion 
No Opinion 
No Opinion 


No Opinion 


No Opinion 


No Opinion 
No Opinion 
No Opinion 
No Opinion 
No Opinion 


No Opinion 


No Opinion 


No Opinion 
No Opinion 


No Opinion 


No Opinion 
No Opinion 
No Opinion 


No Opinion 





9. Level of knowledge of Personnel/Manpower management 
policies and procedures: 


* theories of manpower management 
* tenets of personnel administration 


10. Level of relevance of basic Personnel/Manpower 
management policies and procedures to the job. 


11. Level of knowledge of training/educational psychology: 
¢ leaming processes 
¢ learmng theory 


12. Level of relevance of Training/Educational Psychology 
to the job. 


13. Level of knowledge of the Training Research and 
Development process: 


* steps for the research and development process 
* research and development initiation and guidance 


14. Level of relevance of the Training Research and 
Development process to the job. 


15. Level of knowledge of the applications of computer 
technology to Education/Training management: 


¢ ADP systems 

* computer theory and system analysis 
¢ management of computer applications 
* computer capability/limitations 

* computer assisted mstruction 


16. Level of relevance of computer technology knowledge to 
the job. 


17. Level of knowledge on Contract Administration: 


* procurement management 


* procurement regulations 


¢ product evaluation 


18, Level of relevance of Contract Administration to the job. 


Very High 


Very High 
Very High 


Very High 


Very High 
Very High 
Very High 


Very High 


Very High 


Very High 
Very High 


Very High 


Very High 


Very High 
Very High 
Very High 
Very High 
Very High 


Very High 


Very High 
Very High 
Very High 
Very High 


Very High 


High 


High 
High 


High 


High 
High 
High 


High 


High 


High 
High 


High 


High 


High 
High 
High 
High 
High 


High 
High 
High 
High 


High 


Moderate 


Moderate 
Moderate 


Moderate 


Moderate 
Moderate 
Moderate 


Moderate 


Moderate 


Moderate 
Moderate 
Moderate 


Moderate 


Moderate 
Moderate 
Moderate 
Moderate 
Moderate 
Moderate 


Moderate 
Moderate 
Moderate 
Moderate 
Moderate 


Low 


Low 


Low 


Very Low 


Very Low 
Very Low 


Very Low 


Very Low 
Very Low 
Very Low 


Very Low 


Very Low 


Very Low 
Very Low 


Very Low 


Very Low 


Very Low 
Very Low 
Very Low 
Very Low 
Very Low 


Very Low 


Very Low 
Very Low 
Very Low 
Very Low 


Very Low 


No Opinion 


No Opinion 
No Opinion 


No Opinion 


No Opinion 
No Opinion 
No Opinion 


No Opinion 


No Opinion 


No Opinion 
No Opinion 


No Opinion 


No Opinion 


No Opinion 
No Opinion 
No Opinion 
No Opinion 
No Opinion 


No Opinion 


No Opinion 
No Opinion 
No Opinion 
No Opinion 


No Opinion 





19. Level of knowledge on existing and conceptual training Very High High Moderate Low Very Low No Opinion 
systems: 


¢ non ISD training theories: assets/limitations Very High High Moderate Low’ Very Low No Opinion 


20. Level of relevance of knowledge on existing and Very High High Moderate Low Very Low No Opinion 
conceptual training systems to the job. 
21. Do you know what university your XX37P officer attended during his/her participation in the ETMS program? 


22. Do you know if your XX37P officer is/has pursued continuing education since graduation? 


° If yes, do you know what courses and in what field? 
23. What do you feel is more career enhancing for an XX37P officer for after graduation; a department head or utilization tour? 


24. Is there anything that positively stands out about your XX37P officer's knowledge and ability? 


25. Is there anything that negatively stands out about your XX37P officer's knowledge and ability?? 





| 


26. If you could, what changes/improvements would you make to the ETMS program, or curriculum to make it more job 
televant? 


—— eee eee _—_—eeeeeeeeeeeSESs 


Please feel free to make any additional comments. 


Om re eee ee See -—_—o_o— 


ank vou afain for your help. 





Appendix C. - Questionnaire and Interview Open Response Summary 
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Positive ETMS Program Responses 


Interesting subject variety and selection 

Good adult learning and educational psychology courses 

Outstanding ISD instruction 

Business courses were the most valuable 

Graduates have an extensive background in Educational Training and Management 
PPBS course offered was relevant and useful 

Practicum for Principals was the best course 

It made me computer literate (finally) 

Outstanding instructors 

ODU program is more adult learning/HRD thus more relevant than other programs 
Economics of Education course was great 

Teacher Evaluation course was very relevant for Navy instructor evaluations 
Good educational administration, personnel and organizational change courses 
Statistics course was very useful to my current job. 

Excellent cognitive learning theory 


Negative ETMS Program Responses 


University ETMS program advisor is not helpful 

Courses geared strictly to civilian programs/needs (primarily teachers and principals) 
Many course options in the program are not applicable (ie. School Law, Computer 
Programming, Client/consultant courses 

No courses on Contract Administration - need for job 

Management courses redundant for Naval Officers 

Frequently courses most relevant to ESR's no offered (during program completion time) 
Most of what we learned is not relevant or used on the job 

XX37P officers not fully utilized in fleet jobs 

Insufficient time to complete the program 

Civilian instructors have no experience with Navy Education an Training 

ETMS curriculum is disorganized 

No mentors available 

Business courses very hard to enroll in 





Saas Ssa 


Recommended Change Responses to Improve ETMS Program 


Add courses on: 
Contract Administration 
Program evaluation 
How Navy training courses curriculum are designed and developed 
ISD (specially test development and objective writing 
Navy Training ESR course 
Educational Technology (development and training aids) 
3 or 4 elective courses for area of concentration (ED or HRD) 
Training research and development 
Front end analysis 
Allow more time for program completion 


General Respondent Recommendations for XX37's 


Create a specific XX37 career progression 

Ensure first tour is a utilization tour 

Ensure First tour is a "schools house” tour 

CNET should track XX37's to ensure maximum utilization tours 





Appendix D. - Cover Letters 


9 April 1991 


Dear Sir, 


J am a graduate student in the Education and Training Management Subspecialty (ETMS) program 
at San Diego State University. Currently I am working on my masters project, an evaluation of the ETMS 
program, for Dr. J.D. Smith at the office of the Chief of Naval Education and Training (CNET). The focus 
of this evaluation is to determine the level to which various civilian universities’ curricula help ETMS 
subspecialists (XX37P coded officers) acquire knowledge from the ten education skill areas defined by 
CNET, and the relevance and applicability of this knowledge to an XX37P officers job in the fleet. Your 
input is vital to this evaluation. 


Enclosed is a short questionnaire, and a return addressed envelope. It should take approximately 10 
to 15 minutes to complete. If you do not directly oversee the XX37P officer at you command, please ask 
the responsible department head to complete and return it as soon as possible. If there are more than one 
XX37P officer at you command, if you wish, please make copies of this questionnaire to accommodate 
them. If you would like a copy of the finished evaluation summary, just write a note at the end of the 
questionnaire. 


This evaluation will not be valid without your input. Thank you in advance for you assistance. 


With your participation, this evaluation will help to improve the ETMS program for the fleet and future 
graduates. 


= respectfully, 


Aron L. Gaeta,” 


LT 





9 April 1991 


Dear ETMS Subspecialist, 


Iam an ETMS graduate student currently working on my masters project at San Diego State 
University. The project is an evaluation of the ETMS program, for Dr. J.D. Smith at CNET. The focus of 
the evaluation is to determine the level to which the various universities’ curricula helped you acquire the 
knowledge from the ten education skill areas, and the relevance and applicability of this knowledge to your 
job as an XX37P. Your input is vital to this evaluation. 


Enclosed is a short questionnaire, and a return addressed envelope. [t should take you 
approximately 10 to 15 minutes to complete it. If you are not currently ina XX37P coded billet, please 
answer the questions relating them to your last utilization tour. Please return it as soon as possible. If you 
would like a copy of the finished evaluation summary, just write a note at the end of the questionnaire. 


This evaluation will not be valid without your input. Thank you in advance for you assistance and 
expediency. With your participation, this evaluation will help to improve the ETMS program for future 
graduates. 


Very respectfully, 


Cy 


Aron L. Gladney 
LT USN 





Appendix E. - ESR Course Addition Information 





FALL SEMESTER 1991 
ETMS STUDENTS 


ELS 697 Topics: Perspectives of Navy Training 
Index Number: 58308 


Wednesday, 4:20 - 6:50, Batten Arts & Letters Building, Room 331 


This course is designed to incorporate many of the education and 
training objectives in the Education, Training, and Management 
Subspecialty (ETMS) curriculum offered at ODU into a concise 13 
week, three hour course. A seminar approach will be used with much 
of the material presented by guest lecturers, and tours of local 
training facilities and commands. 


General Objectives: 


1. Develop an understanding of the various innovative appli- 
tions to Navy training and appropriate uses of these 
approaches. 


2. Understand the Navy requirements for training in the 21st 
century and new approaches to meet those needs. 


3. Become familiar with the Navy's ETMS billet availabilities 
and selected career assignments. 


4. Consider potential applications of the ETMS training 
received through the university assignments. 


5. Be aware of selected local facilities and each command's 
approach towards fulfilling its mission and supporting Navy 
training. 


All ETMS students should plan take this course as part of their 36 
hour program. 
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